Feasibility of a walking virtual reality system for rehabilitation: objective and subjective parameters

[EN] Background: Even though virtual reality (VR) is increasingly used in rehabilitation, the implementation of walking navigation in VR still poses a technological challenge for current motion tracking systems. Different metaphors simulate locomotion without involving real gait kinematics, which can affect presence, orientation, spatial memory and cognition, and even performance. All these factors can dissuade their use in rehabilitation. We hypothesize that a marker-based head tracking solution would allow walking in VR with high sense of presence and without causing sickness. The objectives of this study were to determine the accuracy, the jitter, and the lag of the tracking system and its elicited sickness and presence in comparison of a CAVE system. Methods: The accuracy and the jitter around the working area at three different heights and the lag of the head tracking system were analyzed. In addition, 47 healthy subjects completed a search task that involved navigation in the walking VR system and in the CAVE system. Navigation was enabled by natural locomotion in the walking VR system and through a specific device in the CAVE system. An HMD was used as display in the walking VR system. After interacting with each system, subjects rated their sickness in a seven-point scale and their presence in the Slater-Usoh-Steed Questionnaire and a modified version of the Presence Questionnaire. Results: Better performance was registered at higher heights, where accuracy was less than 0.6 cm and the jitter was about 6 mm. The lag of the system was 120 ms. Participants reported that both systems caused similar low levels of sickness (about 2.4 over 7). However, ratings showed that the walking VR system elicited higher sense of presence than the CAVE system in both the Slater-Usoh-Steed Questionnaire (17.6 +/- 0.3 vs 14.6 +/- 0.6 over 21, respectively) and the modified Presence Questionnaire (107.4 +/- 2.0 vs 93.5 +/- 3.2 over 147, respectively). Conclusions: The marker-based solution provided accurate, robust, and fast head tracking to allow navigation in the VR system by walking without causing relevant sickness and promoting higher sense of presence than CAVE systems, thus enabling natural walking in full-scale environments, which can enhance the ecological validity of VR-based rehabilitation applications.The authors wish to thank the staff of LabHuman for their support in this project, especially José Miguel Martínez and José Roda for their assistance. This study was funded in part by Ministerio de Economia y Competitividad of Spain (Project NeuroVR, TIN2013-44741-R and Project REACT, TIN2014-61975-EXP), by Ministerio de Educacion y Ciencia of Spain (Project Consolider-C, SEJ2006-14301/PSIC), and by Universitat Politecnica de Valencia (Grant PAID-10-14).Borrego, A.; Latorre Grau, J.; Llorens Rodríguez, R.; Alcañiz Raya, ML.; Noé, E. (2016). Feasibility of a walking virtual reality system for rehabilitation: objective and subjective parameters. Journal of NeuroEngineering and Rehabilitation. 13:1-9. https://doi.org/10.1186/s12984-016-0174-1S1913Lee KM. Presence. Explicated Communication Theory. 2004;14(1):27–50.Riva G. Is presence a technology issue? Some insights from cognitive sciences. Virtual Reality. 2009;13(3):159–69.Banos RM, et al. Immersion and emotion: their impact on the sense of presence. Cyberpsychol Behav. 2004;7(6):734–41.Llorens R, et al. Tracking systems for virtual rehabilitation: objective performance vs. subjective experience. A practical scenario. Sensors (Basel). 2015;15(3):6586–606.Navarro MD, et al. Validation of a low-cost virtual reality system for training street-crossing. A comparative study in healthy, neglected and non-neglected stroke individuals. Neuropsychol Rehabil. 2013;23(4):597–618.Parsons TD. Virtual reality for enhanced ecological validity and experimental control in the clinical, affective and social neurosciences. Front Hum Neurosci. 2015;9:660.Cameirao MS, et al. Neurorehabilitation using the virtual reality based Rehabilitation Gaming System: methodology, design, psychometrics, usability and validation. J Neuroeng Rehabil. 2010;7:48.Llorens R, et al. Improvement in balance using a virtual reality-based stepping exercise: a randomized controlled trial involving individuals with chronic stroke. Clin Rehabil. 2015;29(3):261–8.Llorens R, et al. Videogame-based group therapy to improve self-awareness and social skills after traumatic brain injury. J Neuroeng Rehabil. 2015;12:37.Fong KN, et al. Usability of a virtual reality environment simulating an automated teller machine for assessing and training persons with acquired brain injury. J Neuroeng Rehabil. 2010;7:19.Levin MF, Weiss PL, Keshner EA. Emergence of virtual reality as a tool for upper limb rehabilitation: incorporation of motor control and motor learning principles. Phys Ther. 2015;95(3):415–25.Llorens R, et al. Effectiveness, usability, and cost-benefit of a virtual reality-based telerehabilitation program for balance recovery after stroke: a randomized controlled trial. Arch Phys Med Rehabil. 2015;96(3):418–25. e2.Cruz-Neira C, et al. Scientists in wonderland: A report on visualization applications in the CAVE virtual reality environment. In: 1993. Proceedings IEEE 1993 Symposium on Research Frontiers in Virtual Reality. 1993.Juan MC, Perez D. Comparison of the levels of presence and anxiety in an acrophobic environment viewed via HMD or CAVE. Presence. 2009;18(3):232–48.Yang YR, et al. Virtual reality-based training improves community ambulation in individuals with stroke: a randomized controlled trial. Gait Posture. 2008;28(2):201–6.Cho KH, Lee WH. Virtual walking training program using a real-world video recording for patients with chronic stroke: a pilot study. Am J Phys Med Rehabil. 2013;92(5):371–84.Darter BJ, Wilken JM. Gait training with virtual reality-based real-time feedback: improving gait performance following transfemoral amputation. Phys Ther. 2011;91(9):1385–94.Yang S, et al. Improving balance skills in patients who had stroke through virtual reality treadmill training. Am J Phys Med Rehabil. 2011;90(12):969–78.Walker ML, et al. Virtual reality-enhanced partial body weight-supported treadmill training poststroke: feasibility and effectiveness in 6 subjects. Arch Phys Med Rehabil. 2010;91(1):115–22.Riley PO, et al. A kinematic and kinetic comparison of overground and treadmill walking in healthy subjects. Gait Posture. 2007;26(1):17–24.Alton F, et al. A kinematic comparison of overground and treadmill walking. Clin Biomech. 1998;13(6):434–40.Lee SJ, Hidler J. Biomechanics of overground vs. treadmill walking in healthy individuals. J Appl Physiol. 2008;104(3).Slater M. Measuring presence: a response to the witmer and Singer presence questionnaire. Presence. 1999;8(5):560–5.Viau A, et al. Reaching in reality and virtual reality: a comparison of movement kinematics in healthy subjects and in adults with hemiparesis. J Neuroeng Rehabil. 2004;1(1):11.Parsons TD, et al. The potential of function-led virtual environments for ecologically valid measures of executive function in experimental and clinical neuropsychology. Neuropsychol Rehabil. 2015;11:1–31. doi: 10.1080/09602011.2015.1109524 .Aravind G, Lamontagne A. Perceptual and locomotor factors affect obstacle avoidance in persons with visuospatial neglect. J Neuroeng Rehabil. 2014;11:38.Darekar A, Lamontagne A, Fung J. Dynamic clearance measure to evaluate locomotor and perceptuo-motor strategies used for obstacle circumvention in a virtual environment. Hum Mov Sci. 2015;40:359–71.Whittle MW. Chapter 4 - Methods of gait analysis. In: Whittle MW, editor. Gait analysis. Edinburgh: Butterworth-Heinemann; 2007. p. 137–75.Hodgson E, et al. WeaVR: a self-contained and wearable immersive virtual environment simulation system. Behav Res Methods. 2015;47(1):296–307.Akizuki H, et al. Effects of immersion in virtual reality on postural control. Neurosci Lett. 2005;379(1):23–6.Thies SB, et al. Comparison of linear accelerations from three measurement systems during "reach & grasp". Med Eng Phys. 2007;29(9):967–72.Fiala M. Designing highly reliable fiducial markers. IEEE Trans Pattern Anal Mach Intell. 2010;32(7):1317–24.Garrido-Jurado S, et al. Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognition. 2014;47(6):2280–92.Kim K, et al. Effects of virtual environment platforms on emotional responses. Comput Methods Programs Biomed. 2014;113(3):882–93.Slater M, Steed A. A virtual presence counter. Presence. 2000;9(5):413–34.Witmer BG, Singer MJ. Measuring presence in virtual environments: a presence questionnaire. Presence Teleop Virt. 1998;7(3):225–40.Martín-Gutiérrez J, et al. Design and validation of an augmented book for spatial abilities development in engineering students. Comput Graph. 2010;34(1):77–91.Lopez-Mir F, et al. Design and validation of an augmented reality system for laparoscopic surgery in a real environment. Biomed Res Int. 2013;2013:758491.Abawi DF, Bienwald J, Dorner R. Accuracy in optical tracking with fiducial markers: an accuracy function for ARToolKit. In: Third IEEE and ACM International symposium on mixed and augmented reality, ISMAR 2004. 2004.Malbezin P, Piekarski W, Thomas BH. Measuring ARTootKit accuracy in long distance tracking experiments. In: The first IEEE International workshop augmented reality toolkit. 2002.Paquette C, Paquet N, Fung J. Aging affects coordination of rapid head motions with trunk and pelvis movements during standing and walking. Gait Posture. 2006;24(1):62–9.Graham JE, et al. Walking speed threshold for classifying walking independence in hospitalized older adults. Phys Ther. 2010;90(11):1591–7.Gorea A. A refresher of the original Bloch’s Law paper (bloch, july 1885). i-Perception. 2015;6:4.Moss JD, Muth ER. Characteristics of head-mounted displays and their effects on Simulator sickness. Hum Factors. 2011;53(3):308–19.Draper MH, et al. Effects of image scale and system time delay on Simulator sickness within head-coupled virtual environments. Hum Factors. 2001;43(1):129–46.Fujisaki W. Effects of delayed visual feedback on grooved pegboard test performance. Front Psychol. 2012;3:61.Keshner EA, et al. Augmenting sensory-motor conflict promotes adaptation of postural behaviors in a virtual environment. Conf Proc IEEE Eng Med Biol Soc. 2011;2011:1379–82.Slaboda JC, Keshner EA. Reorientation to vertical modulated by combined support surface tilt and virtual visual flow in healthy elders and adults with stroke. J Neurol. 2012;259(12):2664–72.Tossavainen T. Comparison of CAVE and HMD for visual stimulation in postural control research. Stud Health Technol Inform. 2004;98:385–7.Akiduki H, et al. Visual-vestibular conflict induced by virtual reality in humans. Neurosci Lett. 2003;340(3):197–200.Duh HBL, et al. Effects of field of view on balance in an immersive environment. In: Virtual Reality, 2001. Proceedings. IEEE. 2001.Krijn M, et al. Treatment of acrophobia in virtual reality: the role of immersion and presence. Behav Res Ther. 2004;42(2):229–39.Mania K, Chalmers A. The effects of levels of immersion on memory and presence in virtual environments: a reality centered approach. Cyberpsychol Behav. 2001;4(2):247–64.Gorini A, et al. The role of immersion and narrative in mediated presence: the virtual hospital experience. Cyberpsychol Behav Soc Netw. 2011;14(3):99–105.Fromberger P, et al. Virtual viewing time: the relationship between presence and sexual interest in androphilic and gynephilic Men. PLoS One. 2015;10(5), e0127156.Slater M, et al. Visual realism enhances realistic response in an immersive virtual environment. IEEE Comput Graph Appl. 2009;29(3):76–84.Nir-Hadad SY, et al. A virtual shopping task for the assessment of executive functions: Validity for people with stroke. Neuropsychol Rehabil. 2015;11:1–26. doi: 10.1080/09602011.2015.1109523 .Vasilyeva M, Lourenco SF. Development of spatial cognition. Wiley Interdiscip Rev Cogn Sci. 2012;3(3):349–62.Banakou D, Groten R, Slater M. Illusory ownership of a virtual child body causes overestimation of object sizes and implicit attitude changes. Proc Natl Acad Sci U S A. 2013;110(31):12846–51.Yee N, Bailenson JN, Ducheneaut N. The proteus effect: implications of transformed digital self-representation on online and offline behavior. Commun Res. 2009;36(2):285–312.Baylor AL. Promoting motivation with virtual agents and avatars: role of visual presence and appearance. Philos Trans R Soc Lond B Biol Sci. 2009;364(1535):3559–65.Clemente M, et al. Assessment of the influence of navigation control and screen size on the sense of presence in virtual reality using EEG. Expert Sys App. 2014;41(4, Part 2):1584–92.Clemente M, et al. An fMRI study to analyze neural correlates of presence during virtual reality experiences. 2013. Interacting with Computers

Augmented 3D hands: a gesture-based mixed reality system for distributed collaboration

Distributed collaborations between two or more participants on a task involving tangible artifacts (e.g., a machine, a patient, a tool) have become increasingly common in recent years due to rapid development in information and communication technologies. In this paper we focus on a specific type of remote-collaboration system where a remote helper guides a local worker using audio communication and hand gestures to perform a repair or a maintenance task. An established ICT approach to supporting this type of collaboration is to provide a shared visual space and some forms of remote gesture. The shared space typically consists of a video capture of the remote workspace which is then displayed on a 2D screen. However, this type of approach has its limitations. Firstly, it does not provide the helper with sufficient understanding of the spatial relationships between objects in the remote workspace. Secondly, it does not allow the helper to gesture in 3D. In an attempt to address these issues, we propose a Mixed Reality multimodal system that improves on previous 2D systems by introducing 3D real-time capturing and rendering of both the remote workspace and the helping hands and by creating a 3D shared visual space as a result of co-locating the remote workspace with the helping hands. In this system, we explore the possibility of increasing the feeling of immersion and co-presence by using head tracking, stereoscopic rendering, inter-occlusion handling and virtual shadowing. In this paper, we introduce HandsIn3D, a system that has been developed for the purpose of the proof of concept. We also present the results of experiments to verify the feasibility of our approach

Evaluating overall quality of dynamic network visualizations

© Springer International Publishing AG 2016. Visualizing dynamic networks is a challenging task. One of the challenges we face is how to maintain visual complexity and overall quality of visualizations at a reasonable and sustainable level so that the information about the network embedded in the visualization can be effectively comprehended by the viewer. Many techniques and algorithms have been proposed and developed to facilitate the discovery of changing patterns. Much research has also been done in investigating how visualization should be constructed to be effective. However, how to measure and compare the quality of visualizations of a changing network at different time points has not been well researched. In this paper, we report on a preliminary work towards this direction. In particular, we apply an existing multi-dimensional overall quality measure in a user study data of different networks and found that the measured quality is positively correlated with user task performance regardless of network size